{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器学习概览\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "机器学习的伟大在于：\n",
    ">1. 对于那些现有解决方案需要大量手动调整或者是规则列表超长的问题： 通过一个机器学习的算法就可以简化代码， 并且提升执行表现。\n",
    "2. 对于那些传统技术手段根本无法解决的复杂问题： 通过最好的机器学习技术可以找到一个解决方案。\n",
    "3. 对于环境波动： 机器学习系统可以适应新的数据。\n",
    "4. 从复杂问题和海量数据中获得洞见。\n",
    "![1-4](img/1-4.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "机器学习系统的种类\n",
    "- 监督式学习\n",
    ">在监督式学习中， 提供给算法的包含所需解决方案的训练数据，称为标签或标记\n",
    ">1. K-近邻算法（k-Nearest Neighbors）\n",
    ">2. 线性回归（Linear Regression）\n",
    ">3. 逻辑回归（Logistic Regression）\n",
    ">4. 支持向量机（Support Vector Machines， SVM）\n",
    ">5. 决策树和随机森林（Decision Trees and Random Forests）\n",
    ">6. 神经网络（Neural networks）\n",
    "![](img/1-5.png)\n",
    "- 无监督式学习\n",
    ">顾名思义， 无监督式学习的训练数据都是未经标记的。 系统会在没有老师的情况下进行学习。\n",
    ">1. 聚类算法\n",
    ">2. k-平均算法（k-Means）\n",
    ">3. 分层聚类分析（Hierarchical Cluster Analysis， HCA）\n",
    ">4. 最大期望算法（Expectation Maximization）\n",
    ">5. 可视化和降维\n",
    ">6. 主成分分析（PCA）\n",
    ">7. 核主成分分析（Kernel PCA）\n",
    ">8. 局部线性嵌入（LLE）\n",
    ">9. t-分布随机近临嵌入（t-SNE）\n",
    ">10. 关联规则学习\n",
    ">11. Apriori\n",
    ">12. Eclat\n",
    "![](img/1-8.png)\n",
    ">\n",
    ">另一个很重要的无监督式任务是异常检测——例如， 检测异常信用卡交易从而防止欺诈， 捕捉制造缺陷， 或者是在提供数据给一种机器学习算法之前， 自动从数据集中移除异常值。\n",
    ">\n",
    "![](img/1-10.png)\n",
    "- 半监督式学习\n",
    ">有些算法可以处理部分标记的训练数据——通常是大量未标记数据和少量的标记数据。\n",
    ">>有些照片托管服务（例如Google相册） 就是很好的例子。 一旦你将所有的家庭照片上传到服务器后， 它会自动识别出人物A出现在照片1、 5和11中， 人物B出现在照片2、 5和7中。 这是算法中无监督的部分（聚类） 。 现在系统需要你做的只是， 告诉它这些人都是谁。 给每个人一个标签之后， 它就可以给每张照片中的每个人命名， 这对于搜索图片非常重要\n",
    "![](img/1-11.png)\n",
    ">\n",
    ">大多数半监督式学习算法是无监督式和监督式算法的结合。例如\n",
    "深度信念网络（DBN） ， 它基于一种互相堆叠的无监督式组件， 这\n",
    "个组件叫作受限玻尔兹曼机（RBM） 。 受限玻尔兹曼机以无监督的\n",
    "方式进行训练， 然后使用监督式学习对整个系统进行微调。\n",
    "- 强化学习\n",
    ">它的学习系统（在其语境中称为智能体） 能够观察环境， 做出选择， 执行操作， 并获得回报（reward） ， 或者是以负面回报的形式获得惩罚。 所以它必须自行学习什么是最好的策略（policy） ， 从而随着时间推移获得最大的回报。 策略代表智能体在特定情况下应该选择的操作。\n",
    "![](img/1-12.png)\n",
    ">\n",
    ">DeepMind的AlphaGo项目也是一个强化学习的好例子\n",
    "- 批量学习\n",
    ">在批量学习中， 系统无法进行增量学习——即必须使用所有可用数据进行训练。 \n",
    ">\n",
    ">如果希望批量学习系统学习新数据（例如新型垃圾邮件） ， 你需\n",
    "要在完整数据集（不仅仅是新数据， 还要包括旧数据） 的基础上重新\n",
    "训练一个新版本的系统， 然后停用旧系统， 用新系统取而代之\n",
    "![](img/1-13.png)\n",
    "- 在线学习\n",
    "> 在在线学习中， 你可以循序渐进地给系统提供训练数据， 逐步积\n",
    "累学习成果。 这种提供数据的方式可以是单独的， 也可以采用小批量\n",
    "（mini-batches） 的小组数据来进行训练。 每一步学习都很快速并且\n",
    "便宜， 所以系统就可以根据飞速写入的最新数据进行学习\n",
    "![](img/1-14.png)\n",
    "- 基于实例的学习\n",
    ">系统先完全记住学习示例（example） ， 然后通过某种相似度度量方式将其泛化到新的实例\n",
    "![](img/1-15.png)\n",
    "- 基于模型的学习\n",
    ">从一组示例集中实现泛化的另一种方法是构建这些示例的模型，然后使用该模型进行预测。\n",
    "![](img/1-16.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepare_country_stats(oecd_bli, gdp_per_capita):\n",
    "    oecd_bli = oecd_bli[oecd_bli['INEQUALITY']=='TOT']\n",
    "    oecd_bli = oecd_bli.pivot(index='Country', columns='Indicator', values='Value')\n",
    "    gdp_per_capita.rename(columns={'2015': 'GDP per capita'}, inplace=True)\n",
    "    gdp_per_capita.set_index('Country', inplace=True)\n",
    "    full_country_stats = pd.merge(left=oecd_bli, right=gdp_per_capita, left_index=True, right_index=True)\n",
    "    full_country_stats.sort_values(by='GDP per capita', inplace=True)\n",
    "    remove_indices = [0, 1, 6, 8, 33, 34, 35]\n",
    "    keep_indices = list(set(range(36)) - set(remove_indices))\n",
    "    return full_country_stats[['GDP per capita', 'Life satisfaction']].iloc[keep_indices]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "oecd_bli = pd.read_csv(\"datasets/lifesat/oecd_bli_2015.csv\", thousands=',')\n",
    "gdp_per_capita = pd.read_csv(\"datasets/lifesat/gdp_per_capita.csv\", thousands=',', delimiter='\\t', encoding='latin1', na_values='n/a')\n",
    "country_stats = prepare_country_stats(oecd_bli, gdp_per_capita)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.c_[country_stats['GDP per capita']]\n",
    "y = np.c_[country_stats['Life satisfaction']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHIpJREFUeJzt3X+UXGWd5/H3p5MmCUkwMQmICT9X5AhuCNgCEdeDMrqCnKAbOMAO4wh7lgVxFHEMeOaMM7izM2uYUX6NBMad2eMvRiEiqICyDAozyI9OSCKMMAKCaYLQ9ISQhqTp0N/94z5dVBfd1dWdvlV1qz6vc+rUrec+9+Z7n3TXt+99nvtcRQRmZmYAHY0OwMzMmoeTgpmZlTgpmJlZiZOCmZmVOCmYmVmJk4KZmZU4KZiZWYmTgpmZlTgpmJlZyfRGBzBRCxcujAMPPLDRYZiZFcq6deteiIhF49UrXFI48MAD6e7ubnQYZmaFIunpWur58pGZmZU4KZiZWYmTgpmZlTgpmJlZiZOCmZmV5JYUJB0qaUPZ6yVJF1bUOV7StrI6X8wrHjMzG19uQ1Ij4jFgGYCkacAzwE2jVL0nIk7OKw4zs6nQ1z9Az9YdLJk/iwVzZkx4fVHU6z6FE4AnIqKmcbJmZs3k5g3PcPHaTXR2dDA4NMTqlUtZsWxxzeuLpF59CmcA14+xbrmkjZJuk3R4neIxM6tJX/8AF6/dxM7BIbYP7GLn4BCr1m6ir3+gpvVFk3tSkLQHsAK4YZTV64EDIuII4CrgB2Ps41xJ3ZK6e3t78wvWzKxCz9YddHaM/Krs7OigZ+uOmtYXTT3OFE4E1kfEc5UrIuKliOhPy7cCnZIWjlLvuojoioiuRYvGnbrDzGzKLJk/i8GhoRFlg0NDLJk/q6b1RVOPpHAmY1w6kvQWSUrLR6d4+uoQk5lZTRbMmcHqlUuZ2dnB3BnTmdnZweqVS0udyeOtLxpFRH47l/YENgMHR8S2VHYeQESskfQp4HxgF7ADuCgi7q22z66urvCEeGZWb0UffSRpXUR0jVsvz6SQBycFawfN/gVTrkixtrNak0Lhps42a3VFGt5YpFitNp7mwqyJFGl4Y5Fitdo5KZg1kSINbyxSrFY7JwWzJlKk4Y1FitVq56Rg1kSKNLyxSLFa7Tz6yKwJFWlET5FibWcefWRWYAvmzCjMF2yRYrXxOSmYFVwR/lLv6x/gkS0vAcHhb31T08ZZTRHaeSo4KZgVWBHuE7h5wzN87nsb2JX6pDunib857Yimi7OaIrTzVHFHs1lBFeE+gb7+AVbduLGUEAAGXws+f2NzxVlNEdp5KjkpmBVUEe4T6Nm6g2l649fMtA41VZzVFKGdp5KTgllBFeE+gSXzZ/FaDL2h/LWhaKo4qylCO08lJwWzgirCfQIL5szgslOPYHrZN03nNHHZqc0VZzVFaOep5PsUzAquCKNiPPqo8XyfglmbKMJ9AgvmzOB9b2/+pyZW++JvdDvXKyk5KZiZ0dzDTusZm/sUzKztNfOw03rH5qRgZm2vmYed1js2JwUza3vNPOy03rE5KZhZ22vmYaf1js1DUs3MkmYedrq7sXlIqpnZBDV62Gk19YrNl4/MzKzEScHMzEqcFMzMrMRJwczMSpwUzMysxEnBzMxKcksKkg6VtKHs9ZKkCyvqSNKVkh6XtEnSUXnFY2aT19c/wMbNLzbFXEDNoJXbI7f7FCLiMWAZgKRpwDPATRXVTgQOSa9jgGvSu5k1iWaePbQRWr096nX56ATgiYh4uqL8FOAbkbkPmCdp3zrFZGbjaObZQxuhHdqjXknhDOD6UcoXA5vLPvekshEknSupW1J3b29vTiGaWaVmnj20EdqhPXJPCpL2AFYAN4y2epSyN0zGFBHXRURXRHQtWtT8T28yaxXNPHtoI7RDe9TjTOFEYH1EPDfKuh5gv7LPS4AtdYjJzGrQzLOHNkI7tEc9JsQ7k9EvHQHcAnxK0j+SdTBvi4hn6xCTmdVoxbLFHPe2hU07e2i9tXp75JoUJO0JfBD4H2Vl5wFExBrgVuAk4HHgFeDsPOMxs8lp5tlDG6GV2yPXpBARrwALKsrWlC0HcEGeMZjZxOT5TIFmfl6BZfw8BTMryXMMfquP728VnubCzIB8x+C3w/j+VuGkYGZAvmPw22F8f6twUjAzIN8x+O0wvr9VOCmYGZDvGPx2GN/fKpQNACqOrq6u6O7ubnQYZi3Lo49ak6R1EdE1Xj2PPjKzEVp5DL6Nz0nBzOrCQ1KLwX0KZpY7D0ktDicFM8udh6QWh5OCmeXOQ1KLw0nBzHLnIanF4Y5mM6uLVp9yulU4KZhZ3Xi4a/Pz5SMzMytxUjAzsxInBTMzK3FSMDOzkpo6miVNA/Yprx8Rv80rKDMza4xxk4KkPwL+DHgOGL77JIClOcZlZmYNUMuZwmeAQyOiL+9gzMyssWrpU9gMbMs7EDMza7xazhSeBH4m6cdAaUrDiPhKblGZmVlD1JIUfptee6SXmZm1qHGTQkRcCiBpbvYx+nOPyszMGmLcPgVJ75T0EPAw8IikdZIOzz80MzOrt1o6mq8DLoqIAyLiAOBzwN/lG5aZmTVCLUlhdkTcNfwhIn4GzK5l55LmSbpR0qOSfiVpecX64yVtk7Qhvb44oejNzGxK1TT6SNKfAt9Mn88CflPj/q8Abo+IUyXtAew5Sp17IuLkGvdnbaavf8Dz70+Q28x2Ry1J4RzgUuD7gIC7gbPH20jSXsD7gE8ARMSrwKuTDdTaz80bnuHitZvo7OhgcGiI1SuXsmLZ4kaH1dTcZra7xr18FBFbI+LTEXFURBwZEZ+JiK017PtgoBf4B0kPSfq6pNEuOy2XtFHSbe7AtmF9/QNcvHYTOweH2D6wi52DQ6xau4m+/oHxN25TbjObCmMmBUmXp/cfSrql8lXDvqcDRwHXRMSRwMvAJRV11gMHRMQRwFXAD8aI5VxJ3ZK6e3t7a/inreh6tu6gs2Pkj2dnRwc9W3c0KKLm5zazqVDt8tFwH8JfT3LfPUBPRNyfPt9IRVKIiJfKlm+V9DVJCyPihYp615GNgqKrqysmGY8VyJL5sxgcGhpRNjg0xJL5sxoUUfNzm9lUGPNMISLWpcVlEfHz8hewbLwdR8TvgM2SDk1FJwD/Wl5H0lskKS0fneLxxHvGgjkzWL1yKTM7O5g7YzozOztYvXKpO06rcJvZVFBE9T+8Ja2PiKMqyh5Kl4TG23YZ8HWy6TGeJOugPh0gItZI+hRwPrAL2EF2P8S91fbZ1dUV3d3d4/3T1iI8kmbi3GY2GknrIqJr3HpjJQVJZwL/FXgvcE/ZqrnAaxHxe1MR6EQ5KZiZTVytSaFan8K9wLPAQuBvysq3A5t2LzwzM2tGYyaFiHgaeFrS7wNbImIngKRZwBLgqbpEaGZmdVPLNBff4/XHcAK8BtyQTzhmZtZItSSF6eluZKB0Z7Kfq2Bm1oJqSQq9klYMf5B0CvBClfpmZlZQtcx9dB7wbUlXk819tBn4eK5RmZlZQ9Ty5LUngGMlzSEbwro9/7DMzKwRajlTQNJHgMOBmekGZCLiSznGZWZmDVDL4zjXkN2F/Edkl49OAw7IOS4zM2uAWjqa3xMRHwe2RsSlwHJgv3zDMjOzRqglKexM769IeiswCByUX0hmZtYotfQp/FDSPOAysucfBPB3uUZlZmYNMWZSkHRaRNwAfCsiXgTWSvoRMDMittUtQjMzq5tql4++kN7XDhdExIATgplZ66p2+ahP0l3AQaM9fjMiVoyyjZmZFVi1pPARsmcsf5ORU2ebmVmLqjZ19qvAfZLeExG9AJI6gDnlz1Y2M7PWUcuQ1Csk7SVpNtkzlh+T9Pmc4zIzswaoJSkcls4MPgrcCuwP/EGuUZmZWUPUkhQ6JXWSJYWbI2KQ7F4FMzNrMbUkhWvJHr05G7hb0gGA+xTMzFrQuEkhIq6MiMURcVJkngbeX4fYDOjrH2Dj5hfp6x9odChm1gaq3dF8VkR8S9JFY1T5Sk4xWXLzhme4eO0mOjs6GBwaYvXKpaxYtrjRYZlZC6t2pjA7vc8d5TUn57jaXl//ABev3cTOwSG2D+xi5+AQq9Zu8hmDmeWq2n0K16bF/xcR/1K+TtJxuUZl9GzdQWdHBzsZKpV1dnTQs3UHC+bMaGBkZtbKaulovqrGMptCS+bPYnBoaETZ4NAQS+bPalBEZtYOqvUpLAfeAyyq6FfYC5iWd2DtbsGcGaxeuZRVFX0KPkswszxVm/toD7K+g+lk/QjDXgJOrWXn6TkMXwfeSXZvwzkR8Yuy9QKuAE4CXgE+ERHrJ3IArWzFssUc97aF9GzdwZL5s5wQzCx31foUfg78XNL/TcNQJ+MK4PaIOFXSHsCeFetPBA5Jr2OAa9L7lOvrHyjkl+uCOTMKFW9RFfXnw2yq1fLktVckXQYcDswcLoyID1TbSNJewPuAT6T6rwKvVlQ7BfhGRATZ5HvzJO0bEc/Wfgjj89BOq8Y/H2avq6Wj+dvAo2TPZb6U7O7mB2vY7mCgF/gHSQ9J+nqaVK/cYmBz2eeeVDZlPLTTqvHPh9lItSSFBRHxf4DBiPh5RJwDHFvDdtPJnsdwTUQcCbwMXFJRR6Ns94Z5lSSdK6lbUndvb28N//Trhod2lhse2mnmnw+zkWpJCoPp/VlJH5F0JLCkhu16gJ6IuD99vpEsSVTW2a/s8xJgS+WOIuK6iOiKiK5FixbV8E+X7dBDO60K/3yYjVRLUvgLSW8CPgf8Mdloos+Ot1FE/A7YLOnQVHQC2fMYyt0CfFyZY4FtU92fMDy0c2ZnB3NnTGdmZ4eHdlqJfz7MRlLWx5vTzqVlZElkD+BJ4GzgdICIWJOGpF4NfJhsSOrZEdFdbZ9dXV3R3V21yqg8usSq8c+HtTpJ6yKia9x64yUFSauBvwB2ALcDRwAXRsS3piLQiZpsUjDLmxOLNbNak0ItQ1I/FBGrJH2MrA/gNOAuoCFJwawZeVirtYqanryW3k8Cro+If88xHrPC8bBWayW1JIUfSnoU6ALulLQI2JlvWGbF4WGt1kpqefLaJcByoCs9n/kVsjuRzQwPa7XWUsuZAhGxNSJeS8svp+GmZoaHtVprqaWj2czG4RltrVU4KZhNEc9oa61g3MtH6W7jsyR9MX3eX9LR+YdWHH39A2zc/KJHmzSI299s6tRypvA1YAj4APAlYDuwFnh3jnEVhsenN5bb32xq1dLRfExEXEAahhoRW8mmrWh7Hp/eWG5/s6lX0yypkqaRprRO9ykMVd+kPXh8emO5/c2mXi1J4UrgJmBvSf8L+GfgL3ONqiA8Pr2x3P5mU2/MpCDpIICI+DawCvgr4FngoxFxQ33Ca24en95Ybn+zqTfmLKlpRr13SbozIk6oc1xjasZZUj07ZmO5/c3GNxWzpHZI+jPg7ZIuqlwZEV/ZnQBbSSuOTy/SF20rtr9Zo1RLCmcAH0115tYnHGsGHuZp1r7GTAoR8RjwZUmbIuK2OsZkDVQ+zHNnGmS2au0mjnvbQv81btYGxkwKks5KT1c7TNI7Ktf78lFrGh7mubNs1PHwME8nBbPWV+3y0ez0PmeUdfk92NkaysM8zdpbtctH16b3SyvXSbowz6CscYaHea6q6FPwWYJZe5jsLKkXAZdPZSDWPDwNtFn7mmxS0JRGUWdFGm45nryOxcM8zdrTZJNCYfsUWmm4ZSsdi5k1h2rTXGyX9NIor+3AW+sY45RppVk1W+lYzKx5VOtobrkb1lppuGUrHYuZNY9aZkltGa003LKVjsXMmkdbJYVWmlWzlY7FzJrHmLOkNqupmCXVo4/MrN1MxSypUxHEU2TPdH4N2FUZkKTjgZuB36Si70fEl/KMCVpruGUrHYuZNV6uSSF5f0S8UGX9PRFxch3iMDOzcbRVn4KZmVWXd1II4KeS1kk6d4w6yyVtlHSbpMNHqyDpXEndkrp7e3vzi9bMrM3lffnouIjYImlv4A5Jj0bE3WXr1wMHRES/pJOAHwCHVO4kIq4DroOsoznnmM3M2lauZwoRsSW9Pw/cBBxdsf6liOhPy7cCnZIW5hmTmZmNLbekIGm2pLnDy8CHgIcr6rxFktLy0SmevrxiMjOz6vK8fLQPcFP6zp8OfCcibpd0HkBErAFOBc6XtAvYAZwRRbtxwsysheSWFCLiSeCIUcrXlC1fDVydVwxmZjYxHpJqZmYlTgpmZlbipGBmZiVOCmZmVuKkYGZmJU4KZmZW4qRgZmYlTgpmZlbipGBmZiVOCmZmVuKkYGZmJU4KZmZW4qRgZmYlTgpmZlbipGBmZiVOCmZmVuKkYGZmJU4KZmZW4qRgZmYlTgpmZlbipNBm+voH2Lj5Rfr6Bxodipk1oemNDsDq5+YNz3Dx2k10dnQwODTE6pVLWbFscaPDMrMm4jOFNtHXP8DFazexc3CI7QO72Dk4xKq1m3zGYGYjOCm0iZ6tO+jsGPnf3dnRQc/WHQ2KyMyakZNCm1gyfxaDQ0MjygaHhlgyf1aDIjKzZuSk0CYWzJnB6pVLmdnZwdwZ05nZ2cHqlUtZMGdGo0MzsybijuY2smLZYo5720J6tu5gyfxZTghm9ga5JgVJTwHbgdeAXRHRVbFewBXAScArwCciYn2eMbW7BXNmOBmY2Zjqcabw/oh4YYx1JwKHpNcxwDXp3czMGqDRfQqnAN+IzH3APEn7NjgmM7O2lXdSCOCnktZJOneU9YuBzWWfe1KZmZk1QN6Xj46LiC2S9gbukPRoRNxdtl6jbBOVBSmhnAuw//775xOpmZnle6YQEVvS+/PATcDRFVV6gP3KPi8Btoyyn+sioisiuhYtWpRXuGZmbS+3pCBptqS5w8vAh4CHK6rdAnxcmWOBbRHxbF4xmZlZdXlePtoHuCkbdcp04DsRcbuk8wAiYg1wK9lw1MfJhqSenWM8ZmY2jtySQkQ8CRwxSvmasuUALsgrhrz19Q/4RjAzaym+o3mSPA21mbWiRt+nUEiehtrMWpWTwiR4Gmoza1VOCpPgaajNrFU5KUyCp6E2s1bljuZJ8jTUZtaKnBR2Q7NOQ+2hsmY2WU4KLcZDZc1sd7hPoYV4qKyZ7S4nhRbiobJmtrucFFqIh8qa2e5yUmghHiprZrvLHc0txkNlzWx3OCm0oGYdKmtmzc+Xj8zMrMRJwczMSpwUzMysxEnBzMxKnBTMzKzEScHMzEoUEY2OYUIk9QJP57DrhcALOey3SNwGbgNwG0BrtsEBEbFovEqFSwp5kdQdEV2NjqOR3AZuA3AbQHu3gS8fmZlZiZOCmZmVOCm87rpGB9AE3AZuA3AbQBu3gfsUzMysxGcKZmZW0lJJQdLfS3pe0sNlZW+WdIekX6f3+alckq6U9LikTZKOKtvmD1P9X0v6w7Lyd0n6ZdrmSkmq7xGOT9J+ku6S9CtJj0j6TCpvm3aQNFPSA5I2pja4NJUfJOn+dDzflbRHKp+RPj+e1h9Ytq8vpPLHJP3nsvIPp7LHJV1S72OslaRpkh6S9KP0ua3aQNJT6Wd1g6TuVNY2vwuTEhEt8wLeBxwFPFxWthq4JC1fAnw5LZ8E3AYIOBa4P5W/GXgyvc9Py/PTugeA5Wmb24ATG33Mo7TBvsBRaXku8G/AYe3UDimuOWm5E7g/Hdv3gDNS+Rrg/LT8SWBNWj4D+G5aPgzYCMwADgKeAKal1xPAwcAeqc5hjT7uMdriIuA7wI/S57ZqA+ApYGFFWdv8LkyqzRodQA4/BAcyMik8BuyblvcFHkvL1wJnVtYDzgSuLSu/NpXtCzxaVj6iXrO+gJuBD7ZrOwB7AuuBY8huRpqeypcDP0nLPwGWp+XpqZ6ALwBfKNvXT9J2pW1T+Yh6zfIClgB3Ah8AfpSOqd3a4CnemBTa8neh1ldLXT4awz4R8SxAet87lS8GNpfV60ll1cp7RilvWukSwJFkfym3VTukyyYbgOeBO8j+qn0xInalKuVxl441rd8GLGDibdNsLgdWAcMP7l5A+7VBAD+VtE7SuamsrX4XJqqdn7w22rW/mER5U5I0B1gLXBgRL1W51NmS7RARrwHLJM0DbgLeMVq19D7RYx3tj6mmagNJJwPPR8Q6SccPF49StWXbIDkuIrZI2hu4Q9KjVeq25O/CRLXDmcJzkvYFSO/Pp/IeYL+yekuALeOULxmlvOlI6iRLCN+OiO+n4rZrB4CIeBH4Gdk14nmShv8QKo+7dKxp/ZuAf2fibdNMjgNWSHoK+EeyS0iX015tQERsSe/Pk/1xcDRt+rtQs0Zfv5rqF2/sU7iMkZ1Kq9PyRxjZqfRAKn8z8BuyDqX5afnNad2Dqe5wp9JJjT7eUY5fwDeAyyvK26YdgEXAvLQ8C7gHOBm4gZGdrJ9MyxcwspP1e2n5cEZ2sj5J1sE6PS0fxOudrIc3+rirtMfxvN7R3DZtAMwG5pYt3wt8uJ1+FybVbo0OYIp/CK4HngUGybL4fyO7Lnon8Ov0PvyfKeBvya41/xLoKtvPOcDj6XV2WXkX8HDa5mrSzX/N9ALeS3YKuwnYkF4ntVM7AEuBh1IbPAx8MZUfTDZa5PH05Tgjlc9Mnx9P6w8u29efpON8jLKRJalN/y2t+5NGH/M47XE8ryeFtmmDdKwb0+uR4Rjb6XdhMi/f0WxmZiXt0KdgZmY1clIwM7MSJwUzMytxUjAzsxInBTMzK3FSsMKStI+k70h6Mk1j8AtJH0vrjpe0Lc0Q+piku9NdvsPb/rmkZ9LsmQ9LWtG4I5kYSbdKmpden2x0PNZanBSskNIUxT8A7o6IgyPiXWQ3XZXfYXpPRBwZEYcCnwaulnRC2fqvRsQy4DTg7yVN2e9DmoY5l9+viDgpsju155HNbmo2ZZwUrKg+ALwaEWuGCyLi6Yi4arTKEbEB+BLwqVHW/QrYBSwsL09nE9+U9E9pHv3/Xrbu85IeTPPuDz+v4UBlz7H4GtnMrPtV7O/dku5V9pyHByTNTdvcI2l9er0n1T0+nd3cJOlfJa0ZTjLpGQELgf8N/Id0tnOZpDmS7kz7+aWkUybRrtbm2nlCPCu2w8m+eCdiPfD5ykJJx5DNJNo7yjZLyaYxmA08JOnHwDuBQ8jm0RFwi6T3Ab8FDiW743XEX/DKHmbzXeD0iHhQ0l7ADrJ5dz4YETslHUJ2V35X2uxosucZPA3cDvwX4May3V4CvDOd7QzPWfSxyCZAXAjcJ+mW8B2qNgFOCtYSJP0t2RQfr0bEu8eqVvH5s5LOAraTfVmP9uV5c0TsAHZIuovsi/q9wIfIptIAmEOWJH4LPB0R942yn0OBZyPiQYCIeCnFPZvsstYy4DXg7WXbPBART6Z616d/90bGJuAvU4IaIpvGeR/gd1W2MRvBScGK6hFg5fCHiLgg/XXcXWWbI4FflX3+akT89Tj/TmWiGJ4y+a8i4tryFen5FS+PsR+Nsi+AzwLPAUeQXc7dOc6/Xc3vk00G+K6IGEwzpM4cZxuzEdynYEX1T8BMSeeXle05VmVJS4E/JZvwbCJOUfbM5wVkE8s9SPb0sXPSMyuQtDjN11/No8BbJb07bTO3bIrqZyNiCPgDshlIhx2t7JnKHcDpwD9X7HM72SNXh72J7BkKg5LeDxwwwWM185mCFVNEhKSPAl+VtIqsP+Bl4OKyav9J0kNkyeJ54NMRcecE/6kHgB8D+wP/M7L5+bdIegfwi/Twon7gLLLLP2PF+6qk04GrJM0i60/4PeBrwFpJpwF3MfJM4xdkncn/Ebib7HkA5fvsk/Qvkh4mm7b5y8APlT2gfgNZIjKbEM+SajYGSX8O9NdwiSmPf/t44I8j4uTx6ppNJV8+MjOzEp8pmJlZic8UzMysxEnBzMxKnBTMzKzEScHMzEqcFMzMrMRJwczMSv4/7VFPZ89S8lYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "country_stats.plot(kind='scatter', x='GDP per capita', y='Life satisfaction')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_new = [[22587]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "线性回归模型预测结果：[[5.96242338]]\n"
     ]
    }
   ],
   "source": [
    "clf1 = LinearRegression()\n",
    "clf1.fit(x, y)\n",
    "print('线性回归模型预测结果：{0}'.format(clf1.predict(X_new)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kNN模型预测结果：[[5.76666667]]\n"
     ]
    }
   ],
   "source": [
    "clf2 = KNeighborsRegressor(n_neighbors=3)\n",
    "clf2.fit(x, y)\n",
    "print('kNN模型预测结果：{0}'.format(clf2.predict(X_new)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-1.3834729014060683\n",
      "-0.0363636363636366\n"
     ]
    }
   ],
   "source": [
    "clf1.fit(x[:20], y[:20])\n",
    "clf2.fit(x[:20], y[:20])\n",
    "print(clf1.score(x[20:], y[20:]))\n",
    "print(clf2.score(x[20:], y[20:]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
